Marginal Boosting 1
نویسندگان
چکیده
AdaBoost produces a linear combination of weak hypotheses. It has been observed in practice that the generalization error of the algorithm continues to improve even after all examples are classified correctly by the current linear combination, i.e. by a hyperplane in feature space where each weak hypothesis is a dimension. The improvement is attributed to the experimental observation that the distances (margins) of the examples to the separating hyperplane are increasing even when the training error is already zero, i.e. all examples are on the correct side of the hyperplane. We give an iterative version of AdaBoost that explicitly maximizes the minimum margin of the examples. We bound the number of iterations and the number of hypotheses used in the final linear combination which approximates the maximum margin hyperplane with a certain precision. This result is shown to be independent from the size of the hypothesis class – even infinite hypothesis classes are allowed.
منابع مشابه
Additive Regression Trees and Smoothing Splines - Predictive Modeling and Interpretation in Data Mining
We suggest a two-phase boosting method, called “additive regression tree and smoothing splines” (ARTSS), which is highly competitive in prediction performance. However, unlike many automated learning procedures, which lack interpretability and operate as a “black box”, ARTSS allows us to (1) estimate the marginal effect smoothly; (2) test the significance of non-additive effects; (3) separate t...
متن کاملPrecautionary Saving and the Marginal Propensity to Consume Out of Permanent Income
The budget constraint requires that, eventually, consumption must adjust fully to any permanent shock to income. Intuition suggests that, knowing this, optimizing agents will fully adjust their spending immediately upon experiencing a permanent shock. However, this paper shows that if consumers are impatient and are subject to transitory as well as permanent shocks, the optimal marginal propens...
متن کاملBoosting in Location Space
Mathematics Subject Classification (2000) Machine Vision and Applications Abstract The goal of object detection is to find objects in an image. An object detector accepts an image and produces a list of locations as (x, y) pairs. Here we introduce a new concept: location-based boosting. Location-based boosting differs from previous boosting algorithms because it optimizes a new spatial loss fun...
متن کاملPlasmid chemokines and colony-stimulating factors enhance the immunogenicity of DNA priming-viral vector boosting human immunodeficiency virus type 1 vaccines.
Heterologous "prime-boost" regimens that involve priming with plasmid DNA vaccines and boosting with recombinant viral vectors have been shown to elicit potent virus-specific cytotoxic T-lymphocyte responses. Increasing evidence, however, suggests that the utility of recombinant viral vectors in human populations will be significantly limited by preexisting antivector immunity. Here we demonstr...
متن کاملBoosted and Linked Mixtures of HMMs for Brain-Machine Interfaces
We propose two algorithms that decompose the joint likelihood of observing multidimensional neural input data into marginal likelihoods. The first algorithm, boosted mixtures of hidden Markov chains (BMs-HMM), applies techniques from boosting to create implicit hierarchic dependencies between these marginal subspaces. The second algorithm, linked mixtures of hidden Markov chains (LMs-HMM), uses...
متن کاملMPCPerm big.tex Precautionary Saving and the Marginal Propensity To Consume Out of Permanent Income
Because the budget constraint implies that consumption must eventually fully adjust to permanent shocks, intuition suggests that consumption-smoothers will have an immediate marginal propensity to consume of one out of permanent shocks. However, this paper shows that if consumers are impatient and experience both transitory and permanent income shocks, the immediate marginal propensity to consu...
متن کامل